Ray Serve

Google Cloud and Anyscale announced optimizations for Ray Serve LLM on Google Kubernetes Engine (GKE) that deliver up to 5x higher throughput and 8x lower latency for large language model inference. T…

Google Cloud demonstrated a multi-cluster inference setup deploying an LLM across two regional GKE clusters using TPU v6e accelerators and GKE managed DRANET for networking. The experiment, documented…

Chan Zuckerberg Biohub released ESMC, ESMFold2, and ESM Atlas under an MIT license this morning, and Sheaf v0.11 shipped support for both model backends within twelve hours. The same-day integration w…